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ABSTRACT 



This paper discusses the results of a study that 
investigated the implementation of Intensive Family Preservation Services 
(IFPS) in Colorado, the validity of the Colorado Client Assessment Record 
-(CCAR) as a measurement, of children ' s. levels of functioning, and the 
identification of the multivariate predictors of placement. The CCAR assesses 
nine, domain- specif ic areas of functioning in children. Site visits were 
conducted at each of the eight IFSP service sites, including interviews with 
26 program directors, supervisory, and service staff. A sample of 5,000 child 
recipients of mental health services tested the efficacy of the CCAR. Results 
indicated: (1) there is need for additional work in specifying the structure 

of the CCAR; (2) prediction of placement was dependent on the time frame 
under consideration, with models differing completely for placement within 3 
months of service, and placement following 3 months of service; and (3) the 
only service variable which was related to placement was the sheer number of 
follow-up services available, which reduced the risk of placement by 26 
percent for each service. (CR) 
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Introduction 



Intensive family preservation services (IFPS) have emerged 
as a service innovation in both the child welfare and 
children's mental health service systems. IFPS interventions | 
involve the provision of intensive, home-based, 
skill-focused services to families whose children are at 
imminent risk of placement. Colorado has implemented 
IFPS as part of its system of care for children with 
: emotional disabilities and their families. Existing research 
; on IFPS focuses on the child welfare system, with primary i 
attention given to the outcome of placement prevention. 
Increasingly, the focus is shifting toward direct measures of i 
; client functioning and its relationship to placement I 

^ prevention. Several studies (Nelson, 1988; Bath, Richey, & 

; Haapala, 1992; Fraser, Pecora, & Lewis, 1991) indicate that j 
i problems in children's functioning are correlates of 
j placement. 
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j For the children's mental health system, child level of 
functioning is an increasingly important variable, with 
! interest driven by (a) theoretical shifts from psychodynamic 
; and behavioral models toward ecological models for 
intervention and research, and (b) measurement issues, 
i moving from diagnostic categories to Level of Functioning 
I (LOF) measures. One such measure is the Colorado Client 
j Assessment Record (CCAR) which assesses nine, 
domain-specific areas of functioning (Ellis, Wackwitz, & 
Foster, 1991). 

I 

I This summary focuses on three areas of investigation: (a) 
description of the mental health family preservation 
intervention, (b) validity of the CCAR as a measurement of 
children's level of functioning, and (c) identification of the 
I multivariate predictors of placement. 
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Method 



Service Context of IFPS in Colorado's Mental Health 
System 

Establishing a complete description of service innovations is 
imperative prior to examination of client outcomes and 
correlates of those outcomes. In addition, existing research 
in family preservation indicates that heterogeneity of service 
evolution is common across sites (Schuerman, Rzepnicki, 
Littell, & Chak, 1993; McDonald & Associates, 1990). Site 
visits were conducted at each of the eight IFPS service sites, 

I including interviews with 26 program directors, supervisory, 
and service staff. Qualitative interview data was gathered in 
the form of taped interviews, each of which followed a 
semi-structured interview guide focusing on service 
• philosophy, intervention models, and agency-community 
contexts. 

The differing environments in which programs operate have 
driven slight divergence in program models, with the 
primary impacts on intervention length and mechanisms for 
program access. These differences are small, however, and 
must be considered in the context of the many similarities 
among service sites. Programs evidenced striking 
{ similarities in philosophies of service and types of 
I interventions offered. The dominant service themes were: 

(a) the power and nature of the helping relationship, (b) the 
strength of the crisis intervention/brief intervention 
foundation, and (c) the commitment to skill-focused 
intervention. 

Factorial Structure of the Colorado Client Assessment 
Record 

Level of functioning measures can provide information 
important to tailoring interventions, predicting success of 
j these interventions, and describing outcomes. The CCAR is 
j administered for all Department of Mental Health (DMH) 
clients, including children; however, no exploration of its 
factorial structure had been done for a child mental health 
population. The CCAR assesses mental health functioning 
in nine domains (Feeling/Mood/Affect, Thinking, 
Medical/Physical, Substance Use, Family, Interpersonal, 
Role Performance, Social Legal, and Self care/basic needs) 
using a combination of rating scales and checklist items. In 
order to explore the utility of this measure, it was 
hypothesized that the nine factor model which forms the 
conceptual organization of the instrument best fits the data 
obtained from DMH clients. 

Each of the nine Level of Functioning (LOF) scales is 
measured on a 0 to 50 point scale. Of 78 problem checklist 
items, 69 are conceptually linked to the specific level of 
functioning scales and describe attributes of the client and 
the client's interaction with their environment. Fourteen 
Personal Problem Profile (PPP) scales were developed from 
these checklist items. 
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these checklist items. 

Subjects 

The sample consisted of a randomly selected group of 5,000 I 
child recipients of mental health services. Data from a 50% 
sample was used for exploratory factor analysis: scores from , 
the remainder were used for confirmatory factor analysis, 
using the EQS structural equation modeling program. 

Results 

Analysis 

The 9 Level of Functioning scales and 14 Personal Problem 
Profile scales were submitted to exploratory factor analysis 
using maximum likelihood factoring and oblimin rotation 
i due to the theoretical likelihood that dimensions of 
functioning are inter-related. Using the 1 .0 cutoff for 
I evaluation of eigenvalues, a seven factor solution emerged 
explaining 49.7% of variance, with all LOF scales factoring 
with their associated PPP scales. Two sets of LOFs and 
I PPPs factor together: (a) Thinking and Self care/Basic 
I needs, and (b) Role performance and Socio-legal. 
j Correlations between factors range from .04 to .43. 

' Because of the theoretical interest in a nine factor model, 

; this solution was also examined (see Figure 1) . This model 
I explains 56.3 % of the variance. All PPP indicators load 
I strongly with their associated LOF scales on a separate 
j factor. No indicators cross load to other factors. Correlations 
i between the factors range from .05 to .46. Both models were 
, carried forward into the confirmatory factor analysis (CFA) 
using EQS (Bender, 1 989). 

I The seven factor model did not fit the data, with a CFI of 
.796. Examination of the modification indices indicated that ‘ 
j upwards of 20 cross loadings of indicators were necessary to 
achieve adequate fit. The nine-factor model was also tested. 
Maximum likelihood factoring was used, and no special 
problems were observed during optimization. This model 
exhibited improved fit, with a CFI of .867, and examination I 
of the modification indices indicated that relatively few 
cross loadings were necessary to achieve fit. Allowing five 
of 23 indicators to cross load produced a model which fit 
the data with a CFI of .900. Three of these cross loadings, 
however, were very small (i.e., less than .2). The 
I standardized structural matrix is presented in Table 1 . 

i I 

I It is important to consider possible reasons for inadequate fit ; 
j of the hypothesized model. The model may simply not fit 
I these data because it does not contain important theoretical 
elements. However, there may also be technical reasons. 

I Monte Carlo studies indicate that "large" models, even 
j when known to fit perfectly, generate fit estimates lower 
I than "small" models. In addition, while large sample sizes 
I are needed to achieve stability in matrices, very large 
* sample sizes impose power constraints on model fitting, as 
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sample sizes impose power constraints on model fitting, as 
do high factor loadings (Gerbing & Anderson, 1993). In this 
model, factor loadings are high, and the sample and model 
sizes are large. 

Given the evolving understanding of model fit in the field of 
structural equation modeling, it would be premature to 
reject the hypothesized 9 factor model based on the .90 
j cutoff for the CFI. These analyses do support the use of the 
existing Level of Functioning scales, which were carried 
i forward into subsequent multivariate analysis. 

i Multivariate Predictors of Placement Outcomes 

While level of functioning data are instructive, LOF is only 
one variable among many that contribute to the impact of an 
intervention for individual families. To identify which 
variables had the most power to predict placement outcomes 
I for the family preservation intervention, we looked at a 
i family preservation sample that consisted of the 3 1 6 
! children who received family preservation services between 
i 1990 and 1992. Data collected included information on 
demographics, child level of functioning (CCAR), service 
I needs, interventions, and placement outcome. Logistic 
■ regression models, using backward selection with a 
! likelihood ratio criteria, were developed to examine the 
: correlates of placement prevention and to construct a 
I predictive model for placement. 

; Results showed that prediction of placement was dependent 
i on the time frame under consideration, with models 
I differing completely for placement within 3 months of 
service, and placement following 3 months of service. 

! During the first 3 months post-termination, children's level 
j of functioning on the CCAR's socio-legal domain, a 
, previous group home placement, and presence of parental 
i mental health service needs, increased the odds of 
i placement (see Table 2 ) . The only service variable which 
i was related to placement was the sheer number of follow-up 
: services available which reduced the risk of placement by 
i 26% for each service. 

j 

For the 3-6 month post-intervention time period ( see Table 
3), two variables were related to reduced risk of placement. 
Families whose children were at risk of foster care (rather i 
than more intensive levels of care) were at 80% less risk of 
placement. Families whose service focused on managing 
! child behavior were at 90% less risk of placement. In 
I contrast, families whose service focused on substance abuse 
! and very basic life-management skills were at greatest risk 
i (5.7 and 10.0 times the risk of other families, respectively). 

} 

I Implications 

I ; 

I Model Fit: Implications for Program Decisions 

j Information from predictive models can be used in two 
' ways at the program level; (a) to make decisions about the 
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ways at the program level: (a) to make decisions about the 
structure of interventions, and (b) to make decisions about 
access to services. In general, risk and protective factors, 
and the models built of them are not sufficiently accurate in 
their prediction of failure to warrant denial of access to 
services. This information is, however, useful in examining 
i the intervention so as to structure it to maximize use of 
protective factors and craft interventions to focus on risk 
factors. 

i This was certainly true for the models which emerged from 
this study. Although the overall models for the separate time i 
periods fit the data relatively well, they did not predict 
placement very well. In both cases, the model did a good 
job of predicting which children would not be placed 
(correctly classifying 99% of these cases), but a very poor 
job of predicting which children would be placed. The latter 
' is, of course, of the most interest for restricting access to 
I services. The predictive model for the 3-6 month time frame 
I was the best, predicting placement at 50% accuracy; 

I however, this resulted in little practical usefulness. 

I Thus, these results are useful in considering which families 
; may be at increased risk of placement for the purpose of 
i designing or re-designing the targeted intervention package 
j for families with these characteristics. These results are not 
i useful in making screening decisions regarding access to 
^ services. 

I 

Several findings are important for program design. Clearly 
j there is need for focus on short and long-term program 
i strategies for prolonging durability of the intervention, 
i Many family preservation programs focus on internal family 
i functioning. However, programs must also address 
■ children's functioning in the legal realm and promote strong 
! connections to follow-up services. As we look at families 
J "down the road" from intervention, it is clear that families 
! facing substance abuse and extreme family disorganization 
; are much less likely to maintain gains, which again places 
i an emphasis on a strong continuum of care and on 
intervention focus in these areas. 

Implications for Future Research 

With regard to the CCAR, there is need for additional work 
in specifying the structure of the instrument. There is also a 
need to examine the concurrent validity of the instrument by 
comparing it to other measures of children's functioning. 
Another important area for future research lies in the 
examination of the instrument's sensitivity to change and the j 
usefulness of change scores in assessing level of functioning i 
I outcomes. ! 

j Additional qualitative work is needed on client experience 
I of the helping relationship. It seems likely that much of the 
j power of this intervention lies in its approach to practice, 

; specifically in the nature of the helping relationship. If 
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service interventions were adequately modeled in 
quantitative ways, the task of measuring the quality of the 
helping relationship must be faced. An approach to measure 
development which is rooted in the experience of those 
involved in the helping relationship is imperative; especially 
when considering the experience of clients. 




Family preservation interventions occupy a key place in the 
continuum of care. There remains a need to move beyond 
placement prevention to direct measures of client change, 
including exploration of changes in children's functioning 
and the relationships between client change and distal 
outcomes. 
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